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ABSTRACT 

We investigate the role of anisotropic feature extraction 
methods for automatic image registration of remotely 
sensed multitemporal images. Building on the classi- 
cal use of wavelets in image registration, we develop an 
algorithm based on shearlets, a mathematical general- 
ization of wavelets that offers increased directional sen- 
sitivity. Initial experimental results on LANDSAT im- 
ages are presented, which indicate superior performance 
of the shearlet algorithm when compared to classical 
wavelet algorithms. 

Index Terms — Image registration, shearlets, wavelets, 
optimization, remotely-sensed, multitemporal, LAND- 
SAT. 

1 Introduction 

The problem of accurately and robustly registering mul- 
titemporal image data is a significant problem in the 
field of remote sensing. Images of the same scene, 
captured by the same sensor, can have severe misalign- 
ment if taken at different times. This can be due to 
seasonal changes, cloud coverage, and differences in 
sensor placement. In order to register two such scenes, 
techniques developed from the mathematical discipline 
of harmonic analysis have been successful. 

In particular, methods based on wavelets have been 
effectively applied to the problem of registering multi- 
temporal, remotely-sensed images |[T|. By decompos- 
ing an image according to a discrete wavelet algorithm, 
an image is reduced to more essential features, which 
are easier to match. It is well-known that wavelets cap- 


ture certain features of an image, such as textures, quite 
well. Their simple and fast implementation through a 
variety of filter schemes has also contributed to the suc- 
cess of wavelets and related algorithms, such as Simon- 
celli steerable filters 10 - 

However, wavelets are known to be isotropic, mean- 
ing they do not effectively represent images that have di- 
rectional features. This theoretical limitation to wavelets 
has been known since the beginning of their study 10, 
and puts limitations on their effectiveness in analyzing 
certain image classes. In particular, images with strong 
edge-like features, including rivers, roads, and moun- 
tains, arc not optimally represented with wavelet-like al- 
gorithms. 

Anisotropic generalizations of wavelets abound. 
In particular, shearlets (4}, ||5] give a fast, optimized, 
and direct ionally-dcpcndcnt decomposition of images, 
which we shall show yield a robust algorithm for the 
registration of multitemporal images. 

2 Background on Wavelets and Shearlets 

In a broad sense, wavelet algorithms decompose an im- 
age with respect to scale and translation. Mathemati- 
cally, for a signal / £ L 2 ([0, 1] 2 ), understood as an ideal 
image signal, and an appropriately chosen wavelet func- 
tion \| /, / may be decomposed as 
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where t] f m>n (x) = 2™ t| r(A m x — n) and A = 

The collection of wavelet coefficients { (/, } me z.nez 2 
describes the behavior of /, our image signal, at differ- 
ent scales (determined by m) and at different transla- 
tions (determined by n). This continuous scheme is 
discretized to work with real, discrete image signals. 

Shearlets generalize wavelets by decomposing with 
respect not just to scale and translation, but also direc- 
tion. Mathematically, given a signal / £ L 2 (M 2 ) and an 
appropriate base function \| i, we may decompose / as 

oo oo 

[ I £ k)Vij,k, (2) 

i=— “ j=-°°kel? 


3 REGISTRATION ALGORITHM DESIGN 

Our image registration algorithm is based on decompos- 
ing the input and reference images using harmonic anal- 
ysis tools. Our ambition is to compare isotropic wavelet- 
like methods to an anisotropic generalization, shearlets. 
To do so, we consider wavelet-like algorithms in the 
form of spline wavelets and Simoncelli steerable pyra- 
mids 10, and the fast shearlet transform Q. These fea- 
tures are then matched using a non-linear least squares 
optimization algorithm, in order to compute the registra- 
tion transformation between the images. 

We summarize our image registration algorithm in 
terms of the four components of image registration de- 
scribed by Brown (H : 
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where: 

• tyi,j,k (X) = 2 4 \| t{B^A 1 x — k ) . 


1 . Search Space: Rotation and translation. 

2. Features : Wavelet features in one case and shear- 
let features in another. 


• A = 


2 0 
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The new matrix B is a shearing matrix, and adds a di- 
rectional component to our decomposition. The shearlet 
coefficients {(f,yij,k)}i.jez,kez 2 describe the behavior 
of / at different scales (determined by /), translations 
(determined by k) and directions (determined by j). 

The role of shearlets in automatic image registra- 
tion can be understood heuristically as follows: shear- 
lets produce sparse, concentrated, and directional fea- 
tures in images, which give the optimization algorithm 
used for image registration sparse, concentrated features 
with which to match. 

We hypothesized these sparse, concentrated features 
would allow us to register multitemporal images that be- 
gin severely misaligned. More precisely, we hypothe- 
sized the sparsity of our shearlet features would produce 
a more robust registration algorithm, compared to one 
based on classical wavelets. Initial results confirming 
this heuristic argument are discussed in section [4[ 


3. Similarity Metric : Unconstrained least squares. 
That is, if Fr and Fj are the reference and input 
features, N the number of relevant pixels, 

the integer coordinate of the ith pixel, and T p the 
transformation associated to parameters p, we 
seek to minimize the similarity metric given by: 

T{p) = U L ( F R(Tp{xi,yi)) -F I {x i ,y i )f . 

iy i = l 

4. Search Strategy: Modified Marquadt-Levenberg 
method of solving non-linear least squares prob- 
lems Q, IfToI . 

4 INITIAL EXPERIMENTAL RESULTS 

To test our wavelet and shearlet algorithms, we regis- 
tered multitemporal images produced by Landsat-5 TM 
and Landsat-7 ETM+ sensors. These images have been 
processed by USGS EROS Data Center, by removing 
artifacts and resampling. The images may be seen in 
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Fig. 1 . Input and Reference Multitemporal Images. 

Figure [T] Notice the large displacement between the 
two images. Indeed, using ENVI software tools, man- 
ual registration was performed on the images to compute 
an RST transformation between them. The manual reg- 
istration parameters are: ( T x ,T y ,Q ) = (103.2,-8.1,0). 
This is an exceptionally large translation in the y direc- 
tion, so this image pair constitutes a severe misalign- 
ment. 

We performed automatic image registration with 
wavelet-like features and shearlet features, and com- 
pared the results to this manual registration. We allowed 
the initial guess of the registration, denoted (T Xo ,T yo , Go), 
to increase towards the true registration, to determine 
the robustness of the algorithms. The further the ini- 
tial guess from the true registration, the more robust a 
registration algorithm is if it converges to the correct 
registration. The results of our registration experiments 
may be seen in Tables [l][2j|3j|4] 
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M’^o) 

Spline Wavelets 

Shearlets 

(0, 

0,0) 

(-1.5, 1.1, -2.4) 

(-•1, 

•3, .1) 

(10, 

-1,0) 

(10.2, -.6, .1) 

(62.6, 33.1, 8 

54) 

(20, 

-2, 0) 

(18.4, -1.8, -1.0) 

(64.8, 30.3, 

1) 

(30, 

-3,0) 

(29.6, -2.7, -.2) 

(103.6, 

-8.2, 

•1) 

(40, 

-4, 0) 

(39.3, -4.5, -1.3) 

(103.6, 

-8.2, 

•1) 

(50, 

-5,0) 

(39.3, 4.0, -1.3) 

(103.6, 

-8.2, 

•1) 

(60, 

-6, 0) 

(62.9, -1.0, -.1) 

(103.6, 

-8.2, 

•1) 

(70, 

-7,0) 

(70.9, -.2, -1.2) 

(103.6, 

-8.2, 

•1) 

(80, 

-8,0) 

(103.5, -8.0, 0) 

(103.6, 

-8.2, 

•1) 

(90, 

-9, 0) 

(103.5, -8.0, 0) 

(103.6, 

-8.2, 

•1) 

(100, 

-10, 0) 

(103.5, -8.0, 0) 

(103.6, 

-8.2, 

•1) 


Table 1 . Comparison of spline wavelets to shearlets. Bold 
entries indicate convergent registrations. The spline wavelets 
algorithm begins to converge at (80,— 8,0), the shearlets al- 
gorithm at (30,— 3,0). 

Tables[l]and[2]indicate superior robustness of shear- 
lets when compared to spline wavelets or Simoncelli 
band-pass features. Table [3] is ambiguous as to which 
of shearlets or Simoncelli low-pass is more robust for 
these multitemporal images, but more subtle choices of 
initial guesses in Table[4]indicate the superior robustness 
of shearlets in this case as well. 

More comprehensive statistical analysis of these 
image registration experiments, as well as experiments 
with other multitemporal images are in progress, and 
shall be presented at the conference. 
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Table 2. Comparison of Simoncelli band-pass features 
to shearlets. Bold entries indicate convergent registrations. 
The Simoncelli band-pass algorithm begins to converge at 
(80,— 8,0), the shearlets algorithm at (30,— 3,0). 
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•1) 


Table 3. Comparison of Simoncelli low-pass features to 
shearlets, first analysis. Bold entries indicate convergent reg- 
istrations. Both the Simoncelli low-pass algorithm and the 
shearlets algorithm begin to converge at (30, —3,0). 
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(T Xo ,T yo ,Qo) 

Sim.-Low 

Shearlets 

(5, -8, ,0) 
(10, -8,0) 
(20, -8, 0) 

(-8.0, -10.9, -12.2) 
(-13.5,2.1,-14.6) 
(14.5, -8.3, -2.6) 

(103.6, -8.1, 0) 
(103.6, -8.2, 0) 
(103.6, -8.2, 0) 


Table 4. Comparison of Simoncelli low -pass features to 
shearlets, second analysis. Bold entries indicate convergent 
registrations. When the T yo parameter is fixed near the cor- 
rect value, shearlets have superior robustness near the critical 
threshold of T Xq = 30. 






